HypDB: Detect, Explain And Resolve Bias in OLAP

On line analytical processing (OLAP) is an essential element of decisionsupport systems. OLAP tools provide insights and understanding needed for improved decision making. However, the answers to OLAP queries can be biased and lead to perplexing and incorrect insights. In this paper, we propose HypDB, a system to detect, explain, and to resolve bias in decision-support queries. We give a simple definition of a biased query, which performs a set of independence tests on the data to detect bias. We propose a novel technique that gives explanations for bias, thus assisting an analyst in understanding what goes on. Additionally, we develop an automated method for rewriting a biased query into an unbiased query, which shows what the analyst intended to examine. In a thorough evaluation on several real datasets we show both the quality and the performance of our techniques, including the completely automatic discovery of the revolutionary insights from a famous 1973 discrimination case. ∗[email protected] †[email protected] ‡[email protected] 1 ar X iv :1 80 3. 04 56 2v 1 [ cs .D B ] 1 2 M ar 2 01 8